System and methods for accelerated data storage and retrieval

ABSTRACT

Systems and methods for providing accelerated data storage and retrieval utilizing lossless and/or lossy data compression and decompression. A data storage accelerator includes one or a plurality of high speed data compression encoders that are configured to simultaneously or sequentially losslessly or lossy compress data at a rate equivalent to or faster than the transmission rate of an input data stream. The compressed data is subsequently stored in a target memory or other storage device whose input data storage bandwidth is lower than the original input data stream bandwidth. Similarly, a data retrieval accelerator includes one or a plurality of high speed data decompression decoders that are configured to simultaneously or sequentially losslessly or lossy decompress data at a rate equivalent to or faster than the input data stream from the target memory or storage device. The decompressed data is then output at rate data that is greater than the output rate from the target memory or data storage device. The data storage and retrieval accelerator method and system may employed: in a disk storage adapter to reduce the time required to store and retrieve data from computer to disk; in conjunction with random access memory to reduce the time required to store and retrieve data from random access memory; in a display controller to reduce the time required to send display data to the display controller or processor; and/or in an input/output controller to reduce the time required to store, retrieve, or transmit data.

This application is a continuation of U.S. patent application Ser. No.12/690,125, filed Jan. 20, 2010, now pending, which is a continuation ofU.S. patent application Ser. No. 11/230,953, filed Sep. 19, 2005, nowabandoned, which is a continuation of U.S. patent application Ser. No.10/628,801, filed Jul. 28, 2003, now abandoned, which is a continuationof U.S. patent application Ser. No. 09/481,243, filed Jan. 11, 2000, nowU.S. Pat. No. 6,604,158, which is a continuation-in-part of U.S.application Ser. No. 09/266,394 filed on Mar. 11, 1999, now U.S. Pat.No. 6,601,104, all of which are incorporated by reference.

BACKGROUND OF THE INVENTION

1. Technical Field

The present invention relates generally to data storage and retrievaland, more particularly to systems and methods for improving data storageand retrieval bandwidth utilizing lossless and/or lossy data compressionand decompression.

2. Description of the Related Art

Information may be represented in a variety of manners. Discreteinformation such as text and numbers are easily represented in digitaldata. This type of data representation is known as symbolic digitaldata. Symbolic digital data is thus an absolute representation of datasuch as a letter, figure, character, mark, machine code, or drawing.

Continuous information such as speech, music, audio, images and videofrequently exists in the natural world as analog information. As iswell-known to those skilled in the art, recent advances in very largescale integration (VLSI) digital computer technology have enabled bothdiscrete and analog information to be represented with digital data.Continuous information represented as digital data is often referred toas diffuse data. Diffuse digital data is thus a representation of datathat is of low information density and is typically not easilyrecognizable to humans in its native form.

There are many advantages associated with digital data representation.For instance, digital data is more readily processed, stored, andtransmitted due to its inherently high noise immunity. In addition, theinclusion of redundancy in digital data representation enables errordetection and/or correction. Error detection and/or correctioncapabilities are dependent upon the amount and type of data redundancy,available error detection and correction processing, and extent of datacorruption.

One outcome of digital data representation is the continuing need forincreased capacity in data processing, storage, and transmittal. This isespecially true for diffuse data where increases in fidelity andresolution create exponentially greater quantities of data. Datacompression is widely used to reduce the amount of data required toprocess, transmit, or store a given quantity of information. In general,there are two types of data compression techniques that may be utilizedeither separately or jointly to encode/decode data: lossy and losslessdata compression.

Lossy data compression techniques provide for an inexact representationof the original uncompressed data such that the decoded (orreconstructed) data differs from the original unencoded/uncompresseddata. Lossy data compression is also known as irreversible or noisycompression. Negentropy is defined as the quantity of information in agiven set of data. Thus, one obvious advantage of lossy data compressionis that the compression ratios can be larger than that dictated by thenegentropy all at the expense of information content. Many lossy datacompression techniques seek to exploit various traits within the humansenses to eliminate otherwise imperceptible data. For example, lossydata compression of visual imagery might seek to delete informationcontent in excess of the display resolution or contrast ratio of thetarget display device.

On the other hand, lossless data compression techniques provide an exactrepresentation of the original uncompressed data. Simply stated, thedecoded (or reconstructed) data is identical to the originalunencoded/uncompressed data. Lossless data compression is also known asreversible or noiseless compression. Thus, lossless data compressionhas, as its current limit, a minimum representation defined by thenegentropy of a given data set.

It is well known within the current art that data compression providesseveral unique benefits. First, data compression can reduce the time totransmit data by more efficiently utilizing low bandwidth data links.Second, data compression economizes on data storage and allows moreinformation to be stored for a fixed memory size by representinginformation more efficiently.

One problem with the current art is that the bandwidth and storagecapacity of existing memory storage devices severely limit theperformance of consumer, entertainment, office, workstation, servers,and mainframe computers for all disk and memory intensive operations.For example, magnetic disk mass storage devices currently employed in avariety of home, business, and scientific computing applications sufferfrom significant seek-time access delays along with profound read/writedata rate limitations. Currently the fastest available (10,000) rpm diskdrives support only a 22 Megabyte per second at a rate (MB/sec). This isin stark contrast to the modern Personal Computer's Peripheral ComponentInterconnect (PCI) Bus's input/output capability of 528 MB/sec andinternal local bus capability of over 1,064 MB/sec. Substantially fasterprocessor, internal local bus memory, and I/O bus bandwidths areexpected in the near future.

Another problem within the current art is that emergent high performancedisk interface standards such as the Small Computer Systems Interface(SCSI-3) and Fibre Channel offer only the promise of higher datatransfer rates through intermediate data buffering in random accessmemory. These interconnect strategies do not address the fundamentalproblem that all modern magnetic disk storage devices for the personalcomputer marketplace are still limited by the same physical mediarestriction of 22 MB/sec. Faster disk access data rates are onlyachieved by the high cost solution of simultaneously accessing multipledisk drives with a technique known within the art as data striping.

Additional problems with bandwidth limitations similarly occur withinthe art by all other forms of sequential, pseudo-random, and randomaccess mass storage devices. Typically mass storage devices includemagnetic and optical tape, magnetic and optical disks, and varioussolid-state mass storage devices. It should be noted that the presentinvention applies to all forms and manners of memory devices includingstorage devices utilizing magnetic, optical, and chemical techniques, orany combination thereof.

SUMMARY OF THE INVENTION

The present invention is directed to systems and methods for providingaccelerated data storage and retrieval by utilizing lossless and lossydata compression and decompression. The present invention provides aeffective increase of the data storage and retrieval bandwidth of amemory storage device. In one aspect of the present invention, a methodfor providing accelerated data storage comprises the steps of receivinga digital data stream at an input data transmission rate which isgreater than a data storage rate of a target storage device, compressingthe digital data stream at a compression rate that increases theeffective data storage rate of the target storage device, and storingthe compressed digital data stream in the target storage device. Thestep of compressing may be performed using lossless data compression,lossy data compression or a combination of lossless and lossy datacompression.

In another aspect of the present invention, the compression processcomprises the steps of reading a first parameter that is indicative of acompression type to be applied to the input digital data stream, andselecting at least one allowable encoder based on the first parameter.

In yet another aspect, the compression process further comprises thestep of reading a second parameter that is indicative of an amount ofinformation loss that is permissible, lossy data compression isselected.

In another aspect of the present invention, a method for providingaccelerated retrieval of stored data comprises the steps of retrieving acompressed digital data stream from a target storage device at a rateequal to a data access rate of the target storage device anddecompressing the compressed data at a decompression rate that increasesthe effective data access rate of the target storage device. The step ofcompressing may be performed using lossless data compression, lossy datacompression or a combination of lossless and lossy data compression.

In yet another aspect of the present invention, the decompressionprocess comprises the steps of reading a first parameter that isindicative of a decompression type to be applied to the compresseddigital data stream, and selecting at least one allowable decoder basedon the first parameter.

In another aspect, the decompression process further comprises the stepof reading a second parameter that is indicative of an amount ofinformation loss that is permissible, if lossy data decompression isselected.

in yet another aspect of the present invention, the method for providingaccelerated data storage utilizes a compression ratio that is at leastequal to the ratio of the input data transmission rate to the datastorage rate so as to provide continuous storage of the input datastream at the input data transmission rate. Moreover, the method forproviding accelerated data retrieval utilizes a decompression ratiowhich is equal to or greater than the ratio of the data access rate to amaximum accepted output data transmission rate so as to provide acontinuous and optimal data output transmission rate.

In another aspect of the present invention, data storage and retrievalacceleration is employed in a disk storage adapter to reduce the timerequired to store and retrieve data from computer to a disk memorydevice.

In another aspect of the present invention, data storage and retrievalacceleration is employed in conjunction with random access memory toreduce the time required to store and retrieve data from random accessmemory.

In another aspect of the present invention, data storage and retrievalacceleration is employed in a video data storage system to reduce thetime required to store digital video data.

In another aspect of the present invention, data storage and retrievalacceleration is employed in a display controller to reduce the timerequired to send display data to the display controller or processor.

In another aspect of the present invention, data storage and retrievalacceleration is employed in an input/output controller to reduce thetime required to store, retrieve, or transmit data various forms ofdata.

The present invention is realized due to recent improvements inprocessing speed, inclusive of dedicated analog and digital hardwarecircuits, central processing units, digital signal processors, dedicatedfinite state machines (and any hybrid combinations thereof), that,coupled with advanced data compression and decompression algorithms, areenabling of ultra high bandwidth data compression and decompressionmethods that enable improved data storage and retrieval bandwidth.

These and other aspects, features and advantages, of the presentinvention will become apparent from the following detailed descriptionof preferred embodiments, that is to be read in connection with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system for accelerated data storage andretrieval according to one embodiment of the present invention;

FIG. 2 is a flow diagram of a method fix accelerated data storage inaccordance with one aspect of the present invention;

FIG. 3 is a flow diagram of a method for accelerated data retrieval inaccordance with one aspect of the present invention;

FIGS. 4 a and 4 b are timing diagrams of methods for accelerated datastorage according to the present invention;

FIGS. 5 a and 5 b are timing diagrams of methods for accelerated dataretrieval according to the present invention;

FIGS. 6 a and 6 b comprise a flow diagram of a method for accelerateddata storage in accordance with a further aspect of the presentinvention;

FIGS. 7 a and 7 b comprise a flow diagram of a method for accelerateddata retrieval in accordance with a further aspect of the presentinvention;

FIG. 8 is a detailed block diagram of a system for accelerated datastorage according to a preferred embodiment of the present invention;

FIG. 9 is a detailed block diagram of a system for accelerated dataretrieval according to a preferred embodiment of the present invention;

FIG. 10 is a block diagram of a system for accelerated video storageaccording to one embodiment of the present invention;

FIG. 11 is a block diagram of a system for accelerated retrieval ofvideo data according to one embodiment of the present invention;

FIG. 12 is a block diagram of an input/output controller system foraccelerated storage of analog, digital, and serial data according to oneembodiment of the present invention;

FIG. 13 is a flow diagram of a method for accelerated storage of analog,digital, and serial data according to one aspect of the presentinvention;

FIG. 14 is a block diagram of an input/output system for acceleratedretrieval of analog, digital, and serial data according to oneembodiment of the present invention; and

FIGS. 15 a and 15 b comprise a flow diagram of method for acceleratedretrieval of analog, digital, and serial data according to one aspect ofthe present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The present invention is directed to systems and methods for providingimproved data storage and retrieval bandwidth utilizing both losslessand lossy data compression and decompression. In the followingdescription, it is to be understood that system elements havingequivalent or similar functionality are designated with the samereference numerals in the Figures. It is to be further understood thatthe present invention may be implemented in various forms of digitaland/or analog hardware, software, firmware, or a combination thereof.Preferably, the present invention is implemented on a computer platformincluding hardware such as one or more central processing units (CPU) ordigital signal processors (DSP), a random access memory (RAM), andinput/output (I/O) interface(s). The computer platform may also includean operating system, microinstruction code; and dedicated processinghardware utilizing combinatorial logic, finite state machines, analog ofsignal processing. The various processes and functions described hereinmay be either part of the hardware, microinstruction code or applicationprograms that are executed via the operating system, or any combinationthereof.

It is to be further understood that, because some of the constituentsystem components described herein are preferably implemented assoftware modules, the actual system connections shown in the Figures maydiffer depending upon the manner in that the systems are programmed. Itis to be appreciated that special purpose microprocessors, digitalsignal processors, analog signal processors, dedicated hardware, or andcombination thereof may be employed to implement the present invention.Given the teachings herein, one of ordinary skill in the related artwill be able to contemplate these and similar implementations orconfigurations of the present invention.

Referring now to FIG. 1, a block diagram illustrates a system foraccelerated data storage and retrieval in accordance with an embodimentof the present invention. The system includes a data storage accelerator10 that is operatively coupled to a data storage device 45. The datastorage accelerator operates to increase the effective data storage rateof the data storage device 45. It is to be appreciated that the datastorage device 45 may be any form of memory device including all formsof sequential, pseudo-random, and random access storage devices. Thememory storage device 45 may be volatile or non-volatile in nature, orany combination thereof Storage devices as known within the current artinclude all forms of random access memory, magnetic and optical tape,magnetic and optical disks, along with various other forms ofsolid-state mass storage devices. Thus it should be noted that thecurrent invention applies to all forms and manners of memory devicesincluding, but not limited to, storage devices utilizing magnetic,optical, and chemical techniques, or any combination thereof

The data storage accelerator 10 receives and processes data blocks froman input data stream. The data blocks may range in size from individualbits through complete files or collections of multiple files, and thedata block size may be fixed variable. In order to achieve continuousdata storage acceleration, the data storage accelerator 10 must beconfigured to compress a given input data block utilizing lossless orlossy data compression at a rate that is equal to or faster than receiptof the input data. Thus, to achieve optimum throughput, the rate thatdata blocks from the input data stream may be accepted by the datastorage accelerator 10 is a function of the size of each input datablock, the compression ratio achieved, and the bandwidth of the targetstorage device. For example, if the data storage device 45 (e.g., atypical target mass storage device) is capable of storing 30 megabytesper second and the data storage accelerator 10 is capable of providingan average compression ratio of 3:1, then 90 megabytes per second may beaccepted as input and the data storage acceleration is precisely 3:1,equivalent to the average compression ratio.

It should be noted that it is not a requirement of the present inventionto configure the storage accelerator 10 to compress a given input datablock at a rate that is equal to or faster than receipt of the inputdata. Indeed, if the storage accelerator 10 compresses data at a ratethat is less than the input data rate, buffering may be applied toaccept data from the input data stream for subsequent compression:Further, since data may be received in high-speed bursts, the presentinvention may increase the effective bandwidth of the data storageprocess without increasing the instantaneous bandwidth of the datastorage device.

Additionally, it is not a requirement that the data storage accelerator10 utilize data compression with a ratio that is at least the ratio ofthe input data stream to the data storage access rate of the datastorage device 45. Indeed, if the compression ratio is less than thisratio, the input data stream may be periodically halted to effectivelyreduce the rate of the input data stream. Alternatively, the input datastream or the output of the data accelerator 10 may be buffered totemporarily accommodate the mismatch in data bandwidth. An additionalalternative is to reduce the input data rate to rate that is equal to orslower than the ratio of the input data rate to the data storage deviceaccess rate by signaling the data input source and requesting a slowerdata input rate, if possible.

Referring again to FIG. 1, a data retrieval accelerator 80 isoperatively connected to and receives data from the data storage device45. The data retrieval accelerator 80 receives and processes compresseddata from data storage device 45 in data blocks that may range in sizefrom individual bits through complete files or collections of multiplefiles. Additionally, the input data block size may be fixed or variable.The data retrieval accelerator 80 is configured to decompress eachcompressed data block which is received from the data storage device 45.In order to achieve continuous accelerated data retrieval, the dataretrieval accelerator must decompress a given input data block at a ratethat is equal to or faster than receipt of the input data.

In a manner analogous to the data storage accelerator 10, achievingoptimum throughput with the data retrieval accelerator 80 is a functionof the rate that compressed data blocks are retrieved from the datastorage device 45, the size of each data block, the decompression ratioachieved, and the limitation on the bandwidth of the output data stream,if any. For example, if the data storage device 45 is capable ofcontinuously supplying 30 megabytes per second and the data retrievalaccelerator 80 is capable of providing an average decompression ratio of1:3, then a 90 megabytes per second output data stream is achieved, andthe corresponding data retrieval acceleration is precisely 1:3,equivalent to the average decompression ratio.

It is to be understood that it is not required that the data retrievalaccelerator so utilize data decompression with a ratio that is at mostequal to the ratio of the retrieval rate of the data storage device 45to the maximum rate data output stream. Indeed, if the decompressionratio is greater than this ratio, retrieving data from the data storagedevice may be periodically halted to effectively reduce the rate of theoutput data stream to be at or below its maximum. Alternatively, thecompressed data retrieved from the data storage device 45 or the outputof the data decompressor may be buffered to temporarily accommodate themismatch in data bandwidth. An additional alternative is to increase theoutput data rate by signaling or otherwise requesting the data outputdevice(s) receiving the output data stream to accept a higher bandwidth,if possible.

Referring now to FIG. 2, a flow diagram of a method for accelerated datastorage according to one aspect of the present invention illustrates theoperation of the data storage acceleration shown in FIG. 1. Aspreviously stated above, data compression is performed on a per datablock basis. Accordingly, the initial input data block in the input datastream (step 200) is input into and compressed by the data storageaccelerator 10 (step 202) utilizing lossless or lossy data compression,or any combination or permutation thereof. Upon completion of theencoding of the input data block, the encoded data block is then storedin the data storage device 45, (step 204). A check or other form of testis performed to see if there are additional data blocks available in theinput stream (step 206). If no more data blocks are available, thestorage acceleration process is terminated (step 208). If more datablocks are available in the input data stream, the next data block isreceived (step 210) and the process repeats beginning with datacompression (step 202).

Referring now to FIG. 3, a flow diagram of a method for accelerated dataretrieval according to one aspect of the present invention illustratesthe operation of the data retrieval accelerator 80 shown in FIG. 1. Datadecompression is also performed on a per data block basis. The initialcompressed data block is retrieved from the storage device 45 (step 300)and is decompressed by the data retrieval accelerator 80 (step 302)utilizing lossless or lossy data decompression, or any combination orpermutation thereof. Upon completion of the decoding of the initial datablock, the decoded data block is then output for subsequent processing,storage, or transmittal (step 304). A check or other form of test isperformed to see if additional data blocks available from the datastorage device (step 306). If no more data blocks are available, thedata retrieval acceleration process is terminated (step 308). If moredata blocks are available from the data storage device, the next datablock is retrieved (step 310) and the process repeats beginning withdata decompression (step 302).

Referring now to FIGS. 4 a and 4 b, a timing diagram illustrates methodsfor accelerated data storage utilizing data compression in accordancewith the present invention. Successive time intervals of equal durationare represented as T1 through T(n+2). Data block 1 is received from aninput stream of one or more data blocks. Similarly, data block 2 throughdata block n are received during time intervals T2 through Tn,respectively. For the purposes of discussion, FIGS. 4 a and 4 bdemonstrate one embodiment of the data storage utilizing a stream of ndata blocks. As previously stated, the input data stream is comprised ofone or more data blocks data blocks that may range in size fromindividual bits through complete files or collections of multiple files.Additionally, the input data block size may be fixed or variable.

In accordance with Method 1, lossless or lossy compression of data block1 and subsequent storage of the encoded data block 1 occurs within timeinterval T1. Similarly, the compression and storage of each successivedata block occurs within the time interval the data block is received.Specifically, data blocks 2 . . . n are compressed time intervals T2 . .. Tn, respectively, and the corresponding encoded data blocks 2 . . . nare stored during the time intervals T2 . . . Tn, respectively. It is tobe understood that Method 1 relies on data compression and encodingtechniques that process data as a contiguous stream (i.e., not blockoriented). It is well known within the current art that certain datacompression techniques including, but not limited to, dictionarycompression, run length encoding, null suppression and arithmeticcompression are capable of encoding data when received. Additionallymany lossy data techniques commonly applied to diffuse data also exhibitthis same capability including, but not limited to adaptive differentialpulse code modulation, linear predictive coding, linear prediction basedanalysis by synthesis coding, subband adaptive transformation andadaptive transform acoustic coding. It is to be appreciated that Method1 possesses the advantage of introducing a minimum delay in the timefrom receipt of input to storage of encoded data blocks.

Referring again to FIGS. 4 a and 4 b, Method 2 illustrates compressingand storing data utilizing pipelined data processing. For Method 2,successive time intervals of equal duration are represented as T1through T(n+2). Data block 1 is received from an input stream of one ormore data blocks during time interval T1. Similarly, data block 2through data block n are received during time intervals T2 through Tn,respectively. Compression of data block 1 occurs during time interval T2and the storage of encoded data block 1 occurs during time interval T3.As shown by Method 2, compression of each successive data block occurswithin the next time interval after the data block is received and datastorage of the corresponding encoded data block occur in the next timeinterval after completion of data compression.

The pipelining of Method 2, as shown, utilizes successive single timeinterval delays for lossless or lossy data compression and data storage.Within the current invention, it is permissible to have increasedpipelining to facilitate additional data processing or storage delays.For example, data compression processing for a single input data blockmay utilize more than one time interval. Accommodating more than onetime interval for data compression requires additional data compressorsto process successive data blocks, e.g., data compression processing ofa single data block through three successive time intervals requiresthree data compressors, each processing a successive input data block.Due to the principle of causality, encoded data blocks are output onlyafter compression encoding.

Method 2 provides for block oriented processing of the input datablocks. Within the current art, block oriented data compressiontechniques provide the opportunity for increased data compressionratios. This includes various forms of dictionary compression, alongwith many compression techniques commonly applied to diffuse image dataincluding current standards by the Joint Photographic Experts Group, theMotion Picture Experts Group, vector quantitization, wavelet coding, andfractal coding. Method 2 may provide increased delay from receipt ofinput data block to storage of encoded data. However, depending onfactors such as the size of input data blocks, the rate that they arereceived, the time required for data compression processing, the datacompression ratio achieved, the bandwidth of the data storage device,and the intended application, the delay may or may not be significant.For example, in a modern database system, recording data for archivalpurposes, the opportunity for increase data compression may far outweighthe need for minimum delay. Conversely, in systems such as a militaryreal-time video targeting system, minimizing delay is often of theessence. It should be noted that Method 1 and Method 2 are not mutuallyexclusive, and may be utilized in any combination.

Referring now to FIGS. 5 a and 5 b, a timing diagram illustrates methodsfor accelerated data retrieval utilizing data decompression inaccordance the present invention shown. Successive time intervals ofequal duration are represented as T1 through T(n+2). Data block 1 isretrieved or otherwise accepted as input from one or more compresseddata blocks retrieved from a data storage device. As shown, data block 2through data block n are retrieved during time intervals T2 through Tn,respectively. For the purposes of discussion, FIGS. 5 a and 5 bdemonstrates one embodiment of the data retrieval accelerator utilizinga stream of a data blocks. Once again, the retrieved data stream iscomprised of one or more data blocks that may range in size fromindividual bits through complete files or collections of multiple files.Additionally, the retrieved data block size may be fixed or variable.

In accordance with Method 1, lossless or lossy decompression of datablock 1 and subsequent outputting of the decoded data block 1 occurswithin time interval T1. Similarly, decompression and outputting of eachsuccessive data block occurs within the time intervals they areretrieved, in particular, data block 2 through data block n aredecompressed and decoded data block 2 through decoded data block n areoutput during time intervals T2 . . . Tn, respectively. It is to beunderstood that Method 1 relies on data decompression and decodingtechniques that process compressed data as a contiguous stream (i.e.,not block oriented). It is well known within the current art thatcertain data decompression techniques including, but not limited to,dictionary compression, run length encoding, null suppression andarithmetic compression are capable of decoding data when received.Method 1 possesses the advantage of introducing a minimum delay in thetime from retrieval of compressed data to output of decoded data blocks.

Referring again to FIGS. 5 a and 5 b, Method 2 involves lossless orlossy decompression and output of data utilizing pipelined dataprocessing. For Method 2, successive time intervals of equal durationare represented as T1 through T(n+2). Data block 1 through data block nare retrieved or otherwise accepted as input from a data storage deviceduring time intervals T1 through Tn, respectively. Decompression of datablock 1 occur during time interval T2 and the decoded data block 1 isoutput during time interval T3. Similarly decompression of eachsuccessive data block occurs within the next time interval after thedata block is retrieved and the outputting of the decoded data blockoccurs during the next time interval after completion of datadecompression.

The pipelining of Method 2, utilizes successive single time intervaldelays for data decompression and data output. Within the currentinvention, it is permissible to have increased pipelining to facilitateadditional data retrieval or data decompression processing delays. Forexample, data decompression processing for a single input data block mayutilize more than one time interval. Accommodating more than one timeinterval for data compression requires additional data decompressors toprocess successive compressed data blocks, e.g., data decompressionprocessing of a single data block through three successive timeintervals requires three data decompressors, each processing asuccessive input data block. Due to the principle of causality, decodeddata blocks are only output after decompression decoding.

As before, Method 2 provides for block oriented processing of theretrieved data blocks. Within the current art, block oriented datadecompression techniques provide the opportunity to utilize bothlossless and lossy data compression encoders that increase datacompression ratios. The disadvantage of method 2 is increased delay fromretrieval of compressed data block to output of decompressed data. Aspreviously discussed for data storage acceleration, depending on thesize of retrieved data blocks, the rate that they are retrieved, thetime required for data decompression processing, the data decompressionratio achieved, the bandwidth of the data output, and the intendedapplication, the delay may or may not be significant.

Referring now to FIGS. 6 a and 6 ba flow diagram illustrates a methodfor accelerated data storage according to a further aspect of thepresent invention. With this method, the lossless or lossy datacompression rate of the storage accelerator 10 is not required to beequal to or greater than the ratio of the input data rate to the datastorage access rate. As previously stated above, data compression isperformed on a per data block basis. Accordingly, the initial input datablock in the input data stream is received (step 600) and then timed andcounted (step 602). Timing and counting enables determination of thebandwidth of the input data stream. The input data block is thenbuffered (step 604).

Optionally, certain data parameters may be read (step 606) to determineWhether the data may be compressed utilizing lossless or lossytechniques. If lossy techniques may be employed, additional parametersmay also be included to indicate the amount of information loss that ispermissible. Allowable encoders and associated parameters are thenselected front the pool of available encoders (step 608). By way ofexample, in one embodiment, header information associated with a givendata block or a series of data blocks may contain a binary flag thatcould be set to either logic “1” or logic “0” to indicate that the givendata block or series of data blocks may be encoded using lossless orlossy data compression, respectively. In another embodiment, amulti-valued encoding parameter may be employed where all values true,for example a 16-bit value of FFFF (hexadecimal), signifies losslessencoding and where each value in the range from FFFE to 0000 denotes theamount of residual information content required. In a furtherembodiment, a list of encoding techniques may be added wherein eachencoding techniques in the list is indexed and selected via using theabove information content values. In yet a further embodiment, thevalues for the information content may possess different meaningsdependent on system context. For example, an incoming video data streammay have an information value of 7FFF. This value may invoke a lossyencoder that scans a system parameter table which indicates videodisplay or printer display resolution. With this information, theencoding algorithm can set the allowed information loss for the encodingprocess. It is to be understood that this technique may be applied toall forms of peripheral input and output devices.

The data is then compressed by the data storage accelerator 10 (step610). During and after the encoding of the input data block, the encodeddata block is then timed and counted (step 612), thus enablingdetermination on of the compression ratio and compression bandwidth. Thecompressed, timed and counted data block is then buffered (step 614).The compression ratio and bandwidths of the input data stream and theencoder are then determined (step 616). The compressed data block isthen stored in the data storage device 45 (step 618). Checks or otherforms of testing are applied to ensure that the data bandwidths of theinput data stream, data compressor, and data storage device arecompatible (step 620). If the bandwidths are not compatible, then one ormore system parameters may be modified to make the bandwidths compatible(step 622). For instance, the input bandwidth may be adjusted by eithernot accepting input data requests, lowering the duty cycle of input datarequests, or by signaling one or more of the data sources that transmitthe input data stream to request or mandate a lower data rate. Inaddition, the data compression ratio of the data storage accelerator 10may be adjusted by applying a different type of encoding process such asemploying lossless or lossy encoding, utilizing a single encoder,multiple parallel or sequential encoders, or any combination thereof todecrease encoding time, increase data compression ratio, or both.Furthermore, additional temporary buffering of either the input datastream or the compressed data stream (or both) may be utilized.

By way of example, assuming the input data rate is 90 MB/sec and thedata storage accelerator 10 provides a compression ration of 3:1, thenthe output of the data storage accelerator 10 would be 30 MB/sec. If themaximum data storage rate of the data storage device 45 is 20 MB/sec(which is less than the data rate output from the data storageaccelerator 10), data congestion and backup would occur at the output ofthe data storage accelerator 10. This problem may be solved by adjustingany one of the system parameters as discussed above, e.g., by adjustingthe compression ratio to provide a data output rate from the datastorage accelerator 10 to be equal to the data storage rate of the datastorage device 45.

On the other hand, if the bandwidths are compatible (or made compatibleby adjusting one or more of the system parameters), then a check orother form of test is performed to determine if there are additionaldata blocks available in the input stream (step 624). If no more datablocks are available, the storage acceleration process is terminated(step 626). If more data blocks are available in the input data stream,the next data block is received (step 628) and the process repeatsbeginning with timing and counting of the input data block (step 602).

Referring now to FIGS. 7 a and 7 b, a flow diagram illustrates a methodfor accelerated data retrieval according to one aspect of the presentinvention. With this method, the data decompression ratio is notrequired to be less than or equal to the ratio of the data retrievalaccess rate to the maximum output data rate. As previously stated above,data decompression is performed on a per data block basis. Accordingly,the initial input data block is retrieved from the storage device (step700) and is timed and counted (step 702). Timing and counting enablesdetermination of the bandwidth of data retrieval. The retrieved datablock is then buffered (step 704). Optionally, encoded or encoded dataparameters may be read (step 706) to select the allowable lossless orlossy decoders and associated data parameters (step 708) using, forexample, the techniques discussed above for the encoding process (e.g.,steps 606 and 608, FIG. 6 a).

Encoded data is then decompressed by the data retrieval accelerator 80(step 710). During and after the decoding the input data block, thedecoded data block is then timed and counted (step 712), thus enablingdetermination of the decompression ratio and decompression bandwidth.The decompressed, timed and counted data block is then buffered (step714). The decompression ratio and bandwidths of the retrieved data andthe decoder are then determined (step 716). The decompressed data blockis then output (step 718). Checks or other forms of testing are appliedto ensure that the data bandwidths of the retrieved data, datadecompressor, and data output are compatible (step 720). If thebandwidths are not compatible, then one or more system parameters may bemodified to make the bandwidths compatible (step 722). For instance, thedata retrieval bandwidth may be adjusted either not accepting(continuously) data blocks retrieved from the data storage device orlowering the duty cycle of data blocks retrieved from the data storagedevice. In addition, one or more of the output data devices that receivethe output data stream may be signaled or otherwise requested to accepta higher data rate. Moreover, a different type of decoding process maybe applied to adjust the data decompression rate by applying, forexample, lossless or lossy decoders, different decoding parameters, asingle decoder, multiple parallel or sequential decoders, or anycombination thereof. Also, additional temporary buffering of either theretrieved or output data or both may be utilized.

By way of example, assuming the data storage device 45 has a dataretrieval rate of 20 MB/sec and the data retrieval accelerator 80provides a 1:4 decompression ratio, then the output of the dataretrieval accelerator 80 would be 80 MB/sec. If the maximum output datatransmission rate that can be accepted from the data retrievalaccelerator 80 is 60 MB/sec (which is lower than the data output datarate of 80 MB/sec of the data retrieval accelerator 80), data congestionand backup would occur at the output of the data retrieval accelerator80. This problem may be solved by adjusting anyone of the systemparameters as discussed above, e.g., by adjusting the decompressionratio to provide a data output rate from the data storage accelerator 80to be equal to the maximum accepted output data transmission rate.

On the other hand, if the bandwidths are compatible (or made compatibleby adjusting one or more system parameters), then a check or other formof test is performed to see if there are additional data blocksavailable from the data storage device (step 724). If no more datablocks are available for output, the retrieval acceleration process isterminated (step 726). If more data blocks are available to be retrievedfrom the data storage device, the next data block is retrieved (step728) and the process repeats beginning with timing and counting of theretrieved data block (return to step 702).

It is to be understood that any conventional compression/decompressionsystem and method (which comply with the above mentioned constraints)may be employed in the data storage accelerator 10 and data retrievalaccelerator 80 for providing accelerated data storage and retrieval inaccordance with the present invention. Preferably, the present inventionemploys the data compression/decompression techniques disclosed in U.S.Ser. No. 09/210,491 entitled “Content Independent Data CompressionMethod and System,” filed on Dec. 11, 1998, which is commonly assignedand which is fully incorporated herein by reference. It is to beappreciated that the compression and decompression systems and methodsdisclosed in U.S. Ser. No. 09/210,491 are suitable for compressing anddecompressing data at rates which provide accelerated data storage andretrieval.

Referring now to FIG. 8, a detailed block diagram illustrates apreferred system for accelerated data storage which employs acompression system as disclosed in the above-incorporated U.S. Ser. No.09/210,491. In this embodiment, the data storage accelerator 10 acceptsdata blocks from an input data stream and stores the input data block inan input buffer or cache 15. It is to be understood that the systemprocesses the input data stream in data blocks that may range in sizefrom individual bits through complete files or collections of multiplefiles. Additionally, the input data block size may be fixed or variable.A counter 20 counts or otherwise enumerate the size of input data blockin any convenient units including bits, bytes, words, double words. Itshould be noted that the input buffer 15 and counter 20 are not requiredelements of the present invention. The input data buffer 15 may beprovided for buffering the input data stream in order to output anuncompressed data stream in the event that, as discussed in furtherdetail below, every encoder fails to achieve a level of compression thatexceeds an a priori specified minimum compression ratio threshold.

Data compression is performed by an encoder module 25 which may comprisea set of encoders E1, E2, E3 . . . En. The encoder set E1, E2, E3 . . .En may include any number “n” (where n may=1) of those lossless encodingtechniques currently well known within the art such as run length,Huffman, Lempel-Ziv Dictionary Compression, arithmetic coding, datacompaction, and data null suppression. It is to be understood that theencoding techniques are selected based upon their ability to effectivelyencode different types of input data. It is to be appreciated that afull complement of encoders are preferably selected to provide a broadcoverage of existing and future data types.

The encoder module 25 successively receives as input each of thebuffered input data blocks (or unbuffered input data blocks from thecounter module 20). Data compression is performed by the encoder module25 wherein each of the encoders E1 . . . En processes a given input datablock and outputs a corresponding set of encoded data blocks. It is tohe appreciated that the system affords a user the option toenable/disable any one or more of the encoders E1 . . . En prior tooperation. As is understood by those skilled in the art, such featureallows the user to tailor the operation of the data compression systemfor specific applications. It is to be further appreciated that theencoding process may be performed either in parallel or sequentially. Inparticular, the encoders El through En of encoder module 25 may operatein parallel (i.e., simultaneously processing a given input data block byutilizing task multiplexing on a single central processor, via dedicatedhardware, by executing on a plurality of processor or dedicated hardwaresystems, or any combination thereof). In addition, encoders E1 throughEn may operate sequentially on a given unbuffered or buffered input datablock. This process is intended to eliminate the complexity andadditional processing overhead associated with multiplexing concurrentencoding techniques on a single central processor and/or dedicatedhardware, set of central processors and/or dedicated hardware, or anyachievable combination. It is to be further appreciated that encoders ofthe identical type may be applied in parallel to enhance encoding speed.For instance, encoder E1 may comprise two parallel Huffman encoders forparallel processing of an input data block.

A buffer counter module 30 is operatively connected to the encodermodule 25 for buffering and counting the size of each of the encodeddata blocks output from encoder module 25. Specifically, thebuffer/counter 30 comprises a plurality of buffer/counters BC1, BC2, BC3. . . BCn; each operatively associated with a corresponding one of theencoders E1 . . . En. A compression ratio module 35, operativelyconnected to the output buffer/counter 30, determines the compressionratio obtained for each of the enabled encoders E1 . . . En by takingthe ratio of the size of the input data block to the size of the outputdata block stored in the corresponding buffer/counters BC1 . . . BCn. Inaddition, the compression ratio module 35 compares each compressionratio with an a priori-specified compression ratio threshold limit todetermine if at least one of the encoded data blocks output from theenabled encoders E1 . . . En achieves a compression that exceeds an apriori-specified threshold. As is understood by those skilled in theart, the threshold limit may he specified as any value inclusive of dataexpansion, no data compression or expansion, or any arbitrarily desiredcompression limit. A description module 38, operatively coupled to thecompression ratio module 35, appends a corresponding compression typedescriptor to each encoded data block which is selected for output so asto indicate the type of compression format of the encoded data block. Adata compression type descriptor is defined as any recognizable datatoken or descriptor that indicates which data encoding technique hasbeen applied to the data. It is to be understood that, since encoders ofthe identical type may be applied in parallel to enhance encoding speed(as discussed above), the data compression type descriptor identifiesthe corresponding encoding technique applied to the encoded data block,not necessarily the specific encoder. The encoded data block having thegreatest compression ratio along with its corresponding data compressiontype descriptor is then output for subsequent data processing, storage,or transmittal. If there are no encoded data blocks having a compressionratio that exceeds the compression ratio threshold limit, then theoriginal unencoded input data block is selected for output and a nulldata compression type descriptor is appended thereto. A null datacompression type descriptor is defined as any recognizable data token ordescriptor that indicates no data encoding has been applied to the inputdata block. Accordingly, the unencoded input data block with itscorresponding null data compression type descriptor is then output forsubsequent data processing, storage, or transmittal.

The data storage acceleration device 10 is connected to a data storagedevice interface 40. The function of the data storage interface 40 is tofacilitate the formatting and transfer of data to one or more datastorage devices 45. The data storage interface may be any of the datainterfaces known to those skilled in the art such as SCSI (SmallComputer Systems Interface), Fibre Channel, “Firewire”. IEEE P1394, SSA(Serial Storage Architecture), IDE (Integrated Disk Electronics), andATA/ATAPI interfaces. It should be noted that the storage device datainterface 40 is not required for implementing the present invention. Asbefore, the data storage device 45 may be any form of memory deviceincluding all forms of sequential, pseudo-random, and random accessstorage devices. The data storage device 45 may be volatile ornon-volatile in nature, or any combination thereof Storage devices asknown within current art include all forms of random access memory(RAM), magnetic and optical tape, magnetic and optical disks, along withvarious other forms of solid-state mass storage devices (e.g., ATA/ATAPIIDE disk). Thus it should be noted that the current invention applies toall forms and manners of memory devices including, but not limited to,storage devices utilizing magnetic, optical, and chemical techniques, orany combination thereof.

Again, it is to be understood that the embodiment of the data storageaccelerator 10 of FIG. 8 is exemplary of a preferred compression systemwhich may be implemented it the preset invention, and that othercompression systems and methods known to those skilled in the art may beemployed for providing accelerated data storage in accordance with theteachings herein. Indeed in another embodiment of the compression systemdisclosed in the above-incorporated U.S. Ser. No. 09/210,491, a timer isincluded to measure the time elapsed during the encoding process againstan a priori-specified time limit. When the time limit expires, only thedata output from those encoders (in the encoder module 25) that havecompleted the present encoding cycle are compared to determine theencoded data with the highest compression ratio. The time limit ensuresthat the real-time or pseudo real-time nature of the data encoding ispreserved. In addition, the results from each encoder in the encodermodule 25 may be buffered to allow additional encoders to besequentially applied to the output of the previous encoder, yielding amore optimal lossless data compression ratio. Such techniques arediscussed in greater detail in the above-incorporated U.S. Ser. No.09/210,491.

Referring now to FIG. 9, a detailed block diagram illustrates apreferred system for accelerated data retrieval employing adecompression system as disclosed in the above-incorporated U.S. Ser.No. 09/210,491. In this embodiment, the data retrieval accelerator 80retrieves or otherwise accepts data blocks from one or more data storagedevices 45 and inputs the data via a data storage interface 50. It is tobe understood that the system processes the input data stream in datablocks that may range in size from individual bits through completefiles or collections of multiple files. Additionally, the input datablock size may be fixed or variable. As stated above, the memory storagedevice 45 may be volatile or non-volatile in nature, or any combinationthereof. Storage devices as known within the current art include allforms of random access memory, magnetic and optical tape, magnetic andoptical disks, along with various other forms of solid-state massstorage devices. Thus it should be noted that the current inventionapplies to all forms and manners of memory devices including storagedevices utilizing magnetic, optical, and chemical techniques, or anycombination thereof. The data storage device interface 50 converts theinput data from the storage device format to a format useful for datadecompression.

The storage device data interface 50 is operatively connected to thedata retrieval accelerator 80 which is utilized for decoding the stored(compressed) data thus providing accelerated retrieval of stored data.In this embodiment, the data retrieval accelerator 80 comprises an inputbuffer 55 which receives as input an uncompressed or compressed datastream comprising one or more data blocks. The data blocks may range insize from individual bits through complete files or collections ofmultiple files. Additionally, the data block size may be fixed orvariable. The input data buffer 55 is preferably included (not required)to provide storage of input data for various hardware implementations. Adescriptor extraction module 60 receives the buffered (or unbuffered)input data block and then parses, lexically, syntactically, or otherwiseanalyzes the input data block using methods known by those skilled inthe art to extract the data compression type descriptor associated withthe data block. The data compression type descriptor may possess valuescorresponding to null (no encoding applied), a single applied encodingtechnique, or multiple encoding techniques applied in a specific orrandom order (in accordance with the data compression system embodimentsand methods discussed above).

A decoder module 65 includes one or more decoders D1 . . . Dn fordecoding the input data block using a decoder, set of decoders, or asequential set of decoders corresponding to the extracted compressiontype descriptor. The decoders D1 . . . Dn may include those losslessencoding techniques currently well known within the art, including runlength, Huffman, Lempel-Ziv Dictionary Compression, arithmetic coding,data compaction, and data null suppression. Decoding techniques areselected based upon their ability to effectively decode the variousdifferent types of encoded input data generated by the data compressionsystems described above or originating from any other desired source.

As with the data compression systems discussed in U.S. application Ser.No. 09/210,491, the decoder module 65 may include multiple decoders ofthe same type applied in parallel so as to reduce the data decodingtime. The data retrieval accelerator 80 also includes an output databuffer or cache 70 for buffering the decoded data block output from thedecoder module 65. The output buffer 70 then provides data to the outputdata stream. It is to be appreciated by those skilled in the art thatthe data retrieval accelerator 80 may also include an input data counterand output data counter operatively coupled to the input and output,respectively, of the decoder module 65. In this manner, the compressedand corresponding decompressed data block may be counted to ensure thatsufficient decompression is obtained for the input data block.

Again, it is to be understood that the embodiment of the data retrievalaccelerator 80 of FIG. 9 is exemplary of a preferred decompressionsystem and method which may be implemented in the present invention, andthat other data decompression systems and methods known to those skilledin the art may be employed for providing accelerated data retrieval inaccordance with the teachings herein.

In accordance with another aspect of the present invention, the datastorage and retrieval accelerator system and method may be employed infor increasing the storage rate of video data. In particular, referringnow to FIG. 10, a block diagram illustrates a system for providingaccelerated video data storage in accordance with one embodiment of thepresent invention. The video data storage acceleration system accepts asinput one or more video data streams that are analog, digital, or anycombination thereof in nature. The input multiplexer 1010 selects theinitial video data stream for data compression and acceleration. Theinput multiplexer 1010 is operatively connected to an A/D converter 1020which converts analog video inputs to digital format of desiredresolution. The A/D converter 1020 may also include functions to stripvideo data synchronization to perform other data formatting functions.It should be noted that the analog-to-digital conversion process is notrequired for digital video inputs. The A/D converter 1020 is operativelyconnected a video memory 1030 that is, in turn, operatively connected toa video processor 1040. The video processor 1040 performs manipulationof the digital video data in accordance with any user desired processingfunctions. The video processor 1040 is operatively coupled to a videooutput memory 1050, that is operatively connected to a data storageaccelerator 10 which compresses the video data to provide acceleratedvideo data to the output data stream for subsequent data processing,storage, or transmittal of the video data. This video data accelerationprocess is repeated for all data blocks in the input data stream. Ifmore video data blocks are available in the input data stream, the videomultiplexer selects the next block of video for accelerated processing.Again, it is to be understood that the data storage accelerator 10 mayemploy any lossless or lossy data compression system which is capable ofcompressing data at a rate suitable for providing accelerated video datastorage in accordance with the teachings herein.

In accordance with another aspect of the present invention, theaccelerated data storage and retrieval system may be employed in adisplay controller to reduce the time required to send display data to adisplay controller or processor. In particular, referring now to FIG.11, a block diagram illustrates a display accelerator system inaccordance with one embodiment of the present invention. The videodisplay accelerator accepts as input one or more digital display datablocks from an input display data stream. It is to be understood thatthe system processes the input data stream in data blocks that may rangein size from individual bits through complete files or collections ofmultiple files. Additionally, the input video data block size may befixed or variable. The input data blocks are processed by a dataretrieval accelerator 80 which employs lossless or lossy datadecompression system in accordance with the teachings herein. Uponcompletion of data decompression, the decompressed data block is thenoutput to a display memory 1110 that provides data to a displayprocessor 1120. The display processor 1120 performs any user desiredprocessing function. It is well known within the current art thatdisplay data is often provided in one or more symbolic formats such asOpen Graphics Language (Open GL) or another display or image language.The display processor 1120 is operatively connected an output memorybuffer 1130. The output memory 1130 supplies data to a display formatter1140 that converts the data to a format compatible with the outputdisplay device or devices. Data from the display formatter 1140 isprovided to the display driver 1150 that outputs data in appropriateformat and drive signal levels to one or more display devices. It shouldbe noted that the display memory 1110, display processor 1120, outputmemory 1130, display formatter 1140, and display driver 1150 are notrequired elements of the present invention.

In accordance with yet another aspect of the present invention, the datastorage and retrieval accelerator system and method may be employed inan I/O controller to reduce the time for storing, retrieving ortransmitting parallel data streams. In particular, referring now to FIG.12, a block diagram illustrates a system for accelerated data storage ofanalog, digital, and serial data in accordance with one embodiment ofthe present invention. The data storage accelerator 10 is capable ofaccepting one or more simultaneous analog, parallel digital, and serialdata inputs. An analog input multiplexer 1205 selects the initial analogdata for data compression and acceleration. The analog input multiplexer1205 is operatively connected to an A/D converter 1210 that converts theanalog input signal to digital data of the desired resolution. Thedigitized data output of the A/D converter 1210 is stored in an analogdata memory buffer 1215 for subsequent data storage acceleration.Similarly, a parallel digital data input multiplexer 1220 selects theinitial parallel digital data for data compression and acceleration. Theparallel digital data input multiplexer 1220 is operatively connected toan input data latch 1225 that holds the input parallel digital data. Theparallel digital data is then stored in digital data memory buffer 1245for subsequent data storage acceleration. In addition, a serial digitaldata input multiplexer 1235 selects the initial serial digital data fordata compression and acceleration. The serial digital data inputmultiplexer 1235 is operatively connected to a serial data interface1240 that converts the serial data stream to a format useful for dataacceleration. The formatted serial digital data is then stored in serialdata memory buffer 1245 for subsequent data acceleration. The analogdata memory 1215, parallel digital data memory 1230, and serial datamemory 1245 are operatively connected to the data storage acceleratordevice 10. Data is selected from each data memory subsystem based upon auser defined algorithm or other selection criteria. It should be notedthat the analog input multiplexer 1205, A/D converter 1210, analog datamemory 1215, parallel data input multiplexer 1220, data latch 1225,digital data memory 1230, serial data input multiplexer 1235, serialdata interface 1240, serial data memory 1245, and counter 20 are notrequired elements of the present invention. As stated above, the datastorage accelerator 10 employs any of the data compression methodsdisclosed in the above incorporated U.S. Ser. No. 09/210,491, or anyconventional lossless or lossy data compression method suitable forcompressing data at a rate necessary for obtaining accelerated datastorage. The data storage accelerator supplies accelerated data to theoutput data stream for subsequent data processing, storage, ortransmittal.

Referring now to FIG. 13, a flow diagram illustrates a method foraccelerated data storage of analog, digital, and serial data accordingto one aspect of the present invention. The analog input multiplexerselects the initial analog data for data compression and acceleration(step 1300). The analog input multiplexer provides analog data to theA/D converter that converts the analog input signal to digital data ofthe desired resolution (step 1302). The digitized data output of the A/Dconverter is then buffered in the analog data memory buffer (step 1304)for subsequent data acceleration. Similarly, the parallel digital datamultiplexer selects the initial parallel digital data for datacompression and acceleration (step 1306). The parallel digital datamultiplexer provides data to the input data latch that then holds theinput parallel digital data (step 1308). The parallel digital data isthen stored in digital data memory buffer for subsequent dataacceleration (step 1310). The serial digital data input multiplexerselects the initial serial digital data for lossless or lossy detailedcompression and acceleration (step 1312). The serial digital data inputmultiplexer provides serial data to the serial data interface thatconverts the serial data stream to a format useful for data acceleration(step 1314). The formatted serial digital data is then stored in theserial data memory buffer for subsequent data acceleration (step 1316).A test or other check is performed to see if new analog data isavailable (step 1318). If no new analog data is available a second checkis performed to see if new parallel data is available (step 1320). If nonew parallel data is available, a third test is performed to see if newserial data is available (step 1322). If no new serial data is available(step 1322) the test sequence repeats with the test for new analog data(step 1318). If new analog data block is available (step 1318), or ifnew parallel data block is available (step 1320), or if new serial datablock is available (step 1322), the input data block is compressed bythe data storage accelerator (step 1324) utilizing any lossless or lossycompression method suitable for providing accelerated data storage inaccordance with the teachings herein. After data compression iscomplete, the compressed data block is then output subsequentaccelerated data processing, storage, or transmittal (step 1326). Afteroutputting data the process repeats beginning with a test for new analogdata (return to step 1318).

Referring now to FIG. 14, a block diagram illustrates system foraccelerated retrieval of analog, digital, and serial data in accordancewith one embodiment of the present invention. A data retrievalaccelerator 80 receives data from an input data stream. It is to beunderstood that the system processes the input data stream in datablocks that may range in size from individual bits through completefiles or collections of multiple files. Additionally, the input datablock size may be fixed or variable. The data retrieval accelerator 80decompresses the input data utilizing any of the lossless or lossydecompression methods suitable for providing accelerate data retrievalin accordance with the teachings herein. The data retrieval accelerator80 is operatively connected to analog data memory 1405, digital datamemory 1420, and serial data memory 1435. Dependent upon the type ofinput data block, the decoded data block is stored in the appropriateanalog 1405, digital 1420, or serial 1435 data memory.

The analog data memory 1405 is operatively connected to a D/A converter1410 that converts the decompressed digital data block into an analogsignal. The D/A converter 1410 is further operatively connected to ananalog hold and output driver 1415. The analog hold and output driver1415 demultiplexer the analog signal output from the D/A converter 1410,samples and holds the analog data, and buffers the output analog data.

In a similar manner, the digital data memory 1420 is operativelyconnected to a digital data demultiplexer 1425 that routes thedecompressed parallel digital data to the output data latch and driver1430. The output latch and driver 1430 holds the digital data andbuffers the parallel digital output.

Likewise, the serial data memory 1435 is operatively connected to aserial data interface 1440 that converts the decompressed data block toan output serial data stream. The serial data interface 1440 is furtheroperatively connected to the serial demultiplexer and driver 1445 thatroutes the serial digital data to the appropriate output and buffers theserial data output.

Referring now to FIGS. 15 a and 15 b, a flow diagram illustrates amethod for accelerated retrieval of analog, digital, and serial dataaccording to one aspect of the present invention. An initial data blockis received (step 1500) and then decompressed by the data storageretrieval accelerator (step 1502) utilizing lossless or lossy datadecompression (as discussed above, for example, with reference to FIGS.7 a and 7 b). Upon completion of data decompression, a test or othercheck is performed to see if the data block is digitized analog data(step 1508). If the data block is not digitised analog data, a secondcheck is performed to see if the data block is parallel digital data(step 1510). If the data block is not parallel digital data, a thirdtest is performed to see if the data block serial data (step 1512). Theresult of at least one of the three tests will be affirmative.

If the data block is comprised of digitized analog data, the decodeddata block is buffered in an “analog” digital data memory (step 1514).The decoded data block is then converted to an analog signal by a D/Aconverter (step 1520). The analog signal is then output (step 1522).

If the data block is comprised of parallel digital data, the decodeddata block is buffered in a “parallel” digital data memory (step 1516).The decoded data block is then demultiplexed (step 1524) and routed tothe appropriate the output data latch and driver. The output latch anddriver then holds the digital data and buffers the parallel digitaloutput (step 1526).

If the data block is comprised of serial data, the decoded data block isbuffered in “serial” digital data memory (step 1518). The decoded datais then formatted to a serial data format (step 1528). The serial datais then demultiplexed, routed to the appropriate output, and output to abuffer (step 1530).

Upon output of analog data (step 1522), parallel digital data (step1526) or serial digital data (step 1530), a test or other form of checkis performed for more data blocks in the input stream (step 1532). If nomore data blocks are available, the test repeats (return to step 1532).If a data, block is available, the next data block is received (step1534) and the process repeats beginning with step 1502.

Although illustrative embodiments have been described herein withreference to the accompanying drawings, it is to be understood that thepresent invention is not limited to those precise embodiments, and thatvarious other changes and modifications may be affected therein by oneskilled in the art without departing from the scope or spirit of theinvention. All such charges and modifications are intended to beincluded within the scope of the invention as defined by the appendedclaims.

What is claimed is:
 1. A program storage device readable by machine,tangibly embodying a software program of instructions executable by atleast one machine to perform a method for providing accelerated datastorage, said method comprising: (a) receiving input digital data at aninput data transmission rate which is greater than an output datatransmission rate to a target storage device, the input digital datacomprising a plurality of data blocks and including disparate datatypes; for respective data blocks in the plurality of data blocks: (b)compressing, using software, the data block with a plurality of encodersto provide a plurality of compressed data blocks; (c) determining acompression ratio associated with each of the plurality of compresseddata blocks; (d) selecting based, at least in part, on the determinedcompression ratios, either the data block or one of the plurality ofcompressed data blocks to include in output digital data, the selectingcomprising: (i) when at least one of the plurality of compressed datablocks has a determined compression ratio that exceeds a compressionratio threshold, selecting a compressed data block having a determinedcompression ratio that exceeds the compression ratio threshold toinclude in the output digital data, and (ii) when none of the pluralityof compressed data blocks has a determined compression ratio thatexceeds the compression ratio threshold, selecting the data block asreceived to include in the output digital data; (e) transmitting theoutput digital data to the target storage device at the output datatransmission rate, wherein the combined length of time required forperforming said compressing and said transmitting the output digitaldata to the target storage device is less than a length of time requiredfor transmitting the input digital data to the target storage device;and (f) adjusting the compression ratio threshold in response to achange in the output data transmission rate.